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The purpose of the study was to gain an understanding of how students viewed the marking 
rubrics that they were expected to follow in the course ARTS 1110 Introduction to University. 
The research questions were “Can first-year students taking the Arts 1110 Introduction to 
University course articulate their understanding of the marking rubric? And, “How do students 
use their understanding of the marking rubric when they are revising their academic essays in 
ARTS 1 1 10?” Data was gathered using paper and pencil surveys that consisted of a combination 
of open-ended and closed questions. The results indicated that students used the rubrics as a way 
to understand where they made mistakes so they could correct them and improve their essays 
either during their rewrite or when they wrote their next essay. The surveys also indicated that 
while the students preferred the hybrid holistic rubric, the analytic rubric was just as effective in 
terms of providing information to the students. 
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Introduction 

Rubrics. We have all used them at some point in our teaching careers. How we 
determine what criteria will be on the rubrics usually comes down to what it is we want to 
measure for that particular task. What we want our students to learn depends on the purpose of 
the assignment. Writing without a clear purpose is difficult, just as writing without clear 
guidelines is also difficult. Designing rubrics is not an exact science and I often find, in my own 
class, when marking essays that I wish I had put in more information or broken down the 
categories into more manageable indices. And, I also have to decide what kind of rubric I should 
use. Should I use analytic rubrics that break down the categories into discrete sets of numbers 
(Moskal, 2000); or do I go with a more holistic measuring device that aims to give a more 
complete picture of the essay (Moskal, 2000)? I also need to think about how much each of the 
categories is worth. I assign more marks to certain areas I determine are more important than to 
lesser areas that are not as important. And, the criteria needs to be made known to the students 
so that they are not left wondering how their essays are going to be marked and what the 
instructor is looking for. Do I expect students to follow the rubric when writing their essays thus 
running the risk of turning the assignments into a formulaic endeavour? And what it is the 
purpose of the rubric? If the rubric is designed to assist students in rewriting their essay then the 
rubric used will be different from the rubric that is designed to tell students why they received 
the mark they did. A good rubric should leave no doubt in the students’ mind as to what they are 
being asked to do on the assignment and the scoring of the indices should provide enough 
breadth to make distinctions between those who fulfilled the criteria to those who only managed 


to get part of the criteria right. 
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Saddler and Andrade (2004) define a rubric as one that “articulates the expectations for 

an assignment by listing the criteria, or what counts and describing the levels of quality” (p. 49). 
They further explain that a rubric should teach as well as evaluate so that students “develop the 
self-regulation skills needed to successfully manage the intricacies of the writing process . . . 
self-directed planning and goal setting, revising, and editing” (p. 49). Similarly, Moskal (2000) 
articulates the benefits of using scoring rubrics in the evaluation process: “(1) they support the 
examination of the extent to which the specified criteria has been reached; and (2) they can 
provide feedback to students concerning how to improve their performance” (p. 4). 

There are two main types of rubrics that are used to score writing assignments, the 
analytic rubric and the holistic rubric. According to De La Paz (2009) and Moskal (2000) the 
analytic rubric looks at each criterion separately as the description of the different score levels 
are developed. While the holistic rubric looks at the collection of criteria throughout the 
construction of each level of the scoring rubric which results in a single descriptive score. De La 
Paz (2009) in her discussion stated that analytic rubrics are useful in the classroom since the 
scoring criteria can help teachers and students identify students’ strengths and weaknesses while 
the holistic rubric is usually used for large-scale assessment because it is thought to be easy to 
use and provides an accurate assessment of the essay. Both types of rubrics are used to maintain 
consistency in marking across students, assignments, as well as between different markers. 

Rezaei and Lovorn (2010) state that “rubrics are regarded as tools that increase reliability 
and validity in assessment” (p. 19). They also point out that teachers’ assessment is more 
reliable if a rubric is used. However, they also caution that in order for rubrics to be useful 
instructors have to be trained in how to use the rubrics otherwise they could end up being little 
more than a checklist. Darling-Hammond (1994) describes rubrics as performance and product- 
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based assessments that provide teachers with information about how well students demonstrate 
their understanding of the task and they can have positive consequences for teaching and 
learning. Similarly, Perlman (2003) defines rubrics as consisting of two parts, a task and a set of 
scoring criteria that lists what students will be assessed on. 

The rubric tells both the instructor and the student what is considered important and what 
to look for when assessing (De La Paz, 2009). De La Paz (2009) further explains that using 
rubrics consistently can be an effective means for guiding student’s revision which may lead to 
improvements in overall writing ability. Livingston (2012) argues that when students learn to 
write they should learn how to meet the expectations of an external audience which is one of the 
benefits of a marking rubric because it allows students to see what they are being assessed on 
and how to improve. Leedback on writing should provide enough guidance that students 
understand what they have to do to improve. Sadler (2009) articulated three purposes for 
rubrics: (1) to formulate standards for achievement; (2) provide an objective way to grade work; 
and, (3) to make expectations clear to students. The rubric provides an objectivity that is not 
found in marking without a rubric since rubrics ensure that teachers have a basis for their final 
assessment. In her discussion of the value of rubrics, Spandel (2006) declared that rubrics “serve 
as a guide to revision, giving student writers an insider’s view of what makes writing work” (p. 
19). She further states that “writing is revision ... if we cannot teach students to revise, we 
cannot, in the truest sense, teach them to write” (p. 20). This was one of the guiding principles 
of this research since the students have the opportunity to rewrite their essays once they have 
been graded. I strive to provide rubrics that not only provide the criteria that students are being 
marked on but also provide guidance so students can revise their essays. 
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Andrade (2001) discussed what she called instructional rubrics. She defined instructional 

rubrics as rubrics that “have been explicitly designed to support as well as to evaluate student 
learning” (p. 13). In her earlier work, Andrade (2000) explained how rubrics can be used as 
teaching tools that support student work and the development of critical thinking skills. She 
identified the features that she sees as necessary to support student learning: 

(l)They [rubrics] are written in language that students can understand, (2) they 
define and describe quality work, (3) they refer to common weaknesses in students’ 
work and indicate how such weaknesses can be avoided, and (4) they can be used by 
students to assess their works-in-progress and thereby guide revision and 
improvement. (Andrade, 2001, p. 1) 

The rubrics that are used in ARTS 1110 can be defined as instructional rubrics because 
they have many of the same characteristics that Andrade (2000) identifies in her criteria. The 
rubrics designed for the course take into account that first year students have to be able to take 
the information from the rubrics and use that information when revising their essays. In order 
for an instructional rubric to be useful it has to provide opportunities for improvement through 
revision. This is one of the key points of the rubrics used in ARTS 1 1 10. Students have the 
opportunity to revise and rewrite their essays once they have received feedback through the use 
of rubrics and teaching assistant comments. Andrade (2001) further indicates that even just the 
process of handing out a rubric and explaining the criteria can increase student knowledge of the 
criteria for writing as communicated by the rubric. 

Similarly Andrade and Bouley (2003) found that students can learn to be self -regulated 
learners through the use of rubrics because an effective rubric allows students to learn through 
the feedback provided as it can direct students to monitor their learning and shows them how to 
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achieve the learning objectives articulated by their instructors. The use of instructional rubrics 
permits students to critically evaluate their own work by focusing on the criteria of what is 
expected in the essays. 

Purpose of the Study 

Like many of my colleagues I spend time examining what has worked in the course and 
what needs to be adjusted from term to term. It is this continuous ability to self-monitor and 
evaluate that allows me to grow as an instructor. One of the areas that I targeted was the 
marking rubric. It has been observed, over the terms that the students did not appear to be 
getting enough information to be able to make significant changes to their essays when they were 
rewriting. I wondered if the rubric that was being used was providing enough guidance to the 
students. I undertook this research in order to test my theory that a hybrid holistic rubric might 
be more effective than the analytical rubric that had been used in the course over the past six plus 
years. 

This study investigated student perceptions of the writing rubrics used in their ARTS 
1110 Introduction to University course. In particular, it attempted to identify the ways in which 
students make use of the marking rubrics when rewriting their argumentative essays. In order to 
better understand what first-year students’ experiences are in learning how to use rubrics when 
writing an academic essay, it was important to listen to the students themselves (Delaney, 2010; 
Groves & Welsh, 2010; MacBeath, 2001; Rohrkemper, 1985; Scherff & Piazza, 2005; Sizer & 
Sizer, 1999; SooHoo, 1993). 

The research questions were “Can first-year students taking the Arts 1110 Introduction 
to University course articulate their understanding of the marking rubric? and, “How do students 
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use their understanding of the marking rubric when they are revising their academic essays in 
ARTS 1110?” 

Participants 

In January 2015, students from three sections of the University of Manitoba’s ARTS 1110 
Introduction to University course were invited to participate in the study by completing surveys 
asking them about the marking rubrics that are used in ARTS 1 1 10. The course has a wide range 
of students enrolled, both in terms of educational background, prior experience with writing, age 
of students, and future career directions. In addition, this course is often taken in the first 
semester of university. 

A total of 83 participants from the control group filled in the survey. The participants in 
the experimental group were required to complete two surveys - 142 participants filled in the 
first survey, and 102 students filled in the second survey. All participants were made aware of 
the parameters of the study before they took part in the study. 

Procedure 

In undertaking this study, the University of Manitoba’s Introduction to University course 
was selected. This is a three-credit course designed to help students make the transition from 
high school to university. It is an interdisciplinary course that provides instruction in learning 
and study strategies, academic writing, research, and critical thinking. The course includes a 
weekly lecture of one hour and fifteen minutes which is taught by an instructor and a weekly 
writing seminar of one hour and fifteen minutes facilitated by a teaching assistant. Each section 
is comprised of approximately 120 students who are divided into seminar groups of 
approximately 30 students. The lecture portion of the course is characterized by lectures on 
concepts and content while the writing seminar sections are characterized by discussions of the 
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application of concepts and opportunities to use those concepts in the students’ own writing 
assignments. There is a direct connection between the content of the lectures and the activities 
in the writing seminars. Students are required to write two 1500 word argumentative essays over 
the course of the term. 

The study involved administering a one-time paper and pencil survey to a control group 
and two paper and pencil surveys to an experimental group. In the 2015 Winter term, students 
from three sections of Arts 1110 were invited to participate in completing a survey(s) that asked 
them questions designed to discover their perceptions of the marking rubrics used to mark their 
argumentative essays. One of the sections served as a control group and had their essays marked 
by an analytic rubric (see Appendix A) that had been used in the course for the past six plus 
years. Two different kinds of rubrics, an analytical and a hybrid holistic, were implemented with 
the two experimental sections. The hybrid holistic rubric (see Appendix B) was designed for this 
study by the author. The three sections were all taught by the author to maintain consistency in 
the expectations and the marking. The teaching of the argumentative essay and the marking of 
the essay are facilitated by teaching assistants who were trained by the course instructor. The 
course instructor also reviewed all of the essays and the marking rubrics and comments to ensure 
that there was a consistency across sections and a reliability in scores. 

Data was collected in two stages over the course of the term. Since the course instructor 
was also the researcher, the teaching assistants were the ones who administered the surveys and 
collected them. The surveys did not include the names of the students so all data is anonymous. 
The teaching assistants put the completed surveys into envelopes that were not opened until the 
course was completed and all final marks had been recorded. 
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Stage 1 

The first survey was administered to the experimental group in the fifth week of the 
Winter 2015 term after the students had received their first marked essay back with teaching 
assistant and instructor feedback on the analytic rubric and on their essay. The survey that was 
administered consisted of ten questions (see Appendix C). This survey was used to obtain 
feedback on how the students perceived the analytic rubric. I was interested in determining if the 
rubric provided students with enough information to understand what they needed to do to 
improve their essays during the rewriting stage. 

Stage 2 

At approximately the twelfth week of the course students from both the control group and 
the experimental group filled in surveys asking them for their opinion on the rubrics used to 
mark their essays. The control group filled in a survey asking for their perceptions of the analytic 
rubric after both marked essays had been returned and they had been permitted to rewrite one of 
the essays. This survey consisted of fifteen questions (see Appendix D). 

The rubric that was used to mark the second essay with the experimental group was a 
hybrid holistic rubric. It is referred to as a hybrid because it used some elements of a holistic 
rubric, in terms of providing an overall mark for the essay without discrete numbering, but it also 
provided criteria for each section (see Appendix E). This second survey consisted of seventeen 
questions. The surveys each took approximately 15 minutes to complete. 

Research Instrument 

The research instruments, appended as Appendix A, B, and C were each onetime paper 
and pencil surveys, analyzed quantitatively. The surveys asked first-year students to give their 
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impressions of the rubrics used to mark their essays over the course of the term. The questions 
on the survey were mainly concerned with the rubric itself in terms of how helpful the rubric was 
both before they wrote their essays and after they received feedback. They also asked students if 
they read the rubrics and why they did or did not read the comments on the rubrics. Additional 
questions asked if the students read the comments provided by the teaching assistants on their 
actual essay and why they did or why they did not. Students were also asked if they went to see 
either their teaching assistants or instructor for help after receiving their essays back. Five of the 
questions were yes/no responses and the rest were open-ended questions where students were 
encouraged to elaborate on their answers. 

Data Analysis 

The surveys were analyzed quantitatively. Each item was analyzed separately using 
frequency distribution. The use of frequency distribution percentages allowed us to determine 
how many students found the rubrics helpful. Trochim (2006) stated that descriptive statistics 
allow a researcher to draw inferences about populations and to estimate the parameters of those 
populations. In addition, the use of descriptive statistics permits some degree of generalization 
from a selected sample to the population. The results reported here are drawn from self-reported 
data and are reported strictly as descriptive percentages. 

Results 

Stage 1 - Experimental Group 

The first survey administered to the experimental group was designed to ask students 
questions about how the analytic rubric met their expectations in terms of allowing them to use 
the information provided to rewrite their essays. One hundred and forty-two students filled in 
the survey. The rubric was made up of ten questions that asked the students to comment on the 
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following: (1) Did the rubric prove helpful when you were writing your essay? Of the responses 
provided 1 17 students said it was helpful while 25 said it was not. (2) Why was it helpful? Or 
why was it not helpful? The majority, 101 students, who found it helpful said that they 
appreciated the rubric because it gave clear expectations of what was expected in their essays, 
helped with organization, and provided a checklist of what was required in each category. A 
further ten students said that the rubric helped them improve their essay before they handed it in. 
Out of the 25 students who said it was not helpful, 13 admitted that they did not read the rubric 
before they wrote their essays. They all said that in hindsight it probably would have been 
helpful to see what was expected. The other 12 said the rubric was not helpful because they 
found parts of it confusing since it did not provide examples of what exactly was expected. (3) 
Students were also asked if they read the comments provided by their teaching assistants. One 
hundred and thirty seven students said they read the comments on the rubric and only four said 
they did not. (4) Why did you or why did you not read the rubric? Thirty-eight students said 
they read the rubric comments because it allowed them to identify their mistakes while 86 
students said they read the comments to see how to improve their essays. There were 1 1 
students who wished that the comments were even more specific than they were. They thought it 
would have helped them if there were examples provided of how they could make changes to 
their essays. (5) Did you read the comments on the essay? One hundred and twenty-nine 
students said they read the comments on the essay while eight students said they did not. (6) 
Why or why didn’t you read the comments on the essay? The answers to this question were 
similar to the answers to the fourth question. Seventy-nine students said they read the comments 
to know how to improve their essays while forty-seven said they read the comments to see what 
mistakes they made so they could avoid making the same errors in subsequent essays they wrote. 



13 


Running Head: UNDERSTANDING RUBRICS 

The students who answered no said they were not planning on rewriting their essays so did not 
bother to read the comments as they were satisfied with their marks, and others found the rubrics 
clear enough. (7) Students were asked if they went to see their teaching assistants when they got 
back their marked essays. Fifty-three students said they did, and 88 said they did not. (8) 
Students were asked why they did or did not go to see their teaching assistants? Twenty-three 
students said they went to clarify comments on the rubrics and on their essays and twenty-three 
said they went because they wanted to improve their grades. Of the students who did not go to 
see their teaching assistants, 46 said there was no need as the rubric and the comments on the 
essay were clear enough to guide them when writing their next essay, ten students cited a lack of 
time, five students relied on emails to answer questions, and five said they asked questions after 
class. (9) Students were also asked if they went to see their instructor after they got their marked 
essays back. Nineteen said yes they had gone to see her while 1 17 said they had not. (10) 
Students were asked why they did or did not go to see their instructor? Thirty-nine students said 
there was no need to go since their teaching assistant had answered all of their questions, twelve 
said they had no time because of their course load, and twenty-three students said the rubric was 
clear enough so there was no need to go. Of the students who did go to see their instructor six 
said they went to clarify their marks, and thirteen went to help improve their understanding of 
what they had to do for the next essay. 

Stage 2 Results 

After the second essay had been marked and handed back, students in the control class, 
who had both essays marked using the analytic rubric, were given a survey to see what they 
thought of the analytic rubric in terms of its usefulness when rewriting their essays. Students in 
the experimental group were provided with the second survey asking them for their thoughts on 
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the two rubrics that had been used in their class, the analytical and the hybrid holistic and to 
compare the two. 

Control Group 

There were 83 students in the control group who filled in the survey. The survey had 15 
questions on it. (1) Did the rubric prove useful when you were writing your essay? Fifty-nine of 
the students said that the rubric was useful and 24 said it was not. (2) Why was it helpful or why 
was it not helpful? Fifty- six of the students said that the rubric was helpful because it showed 
them what needed to be included in their essays, the criteria was specific, and it provided a 
guideline as to what the markers were looking for when grading the essays. Five students 
admitted they did not read the rubric before writing their essays and they said they could see the 
usefulness of the rubric after they got their marked essays back. Twenty of the students found 
the rubric confusing because they felt it was not specific enough. They would have liked to have 
examples provided of what they could include in their essays in terms of how to cite properly, 
how to write an introduction, a conclusion etc. It is worth noting here that the textbook used in 
the course A Student’s Guide to Academic Writing written by O’Brien Moran and Soiferman 
(2014) provides examples of how to craft introductions, conclusions, how to use transitions, how 
to formulate a thesis statement etc. Plus the teaching assistants go over how to structure the 
essay and the elements required in each paper. In other words, there was plenty of opportunity 
for those students to see examples of what was expected for each of the indices. (3) Students 
were asked if they read the comments on the rubric provided by the teaching assistants. Eighty 
two students said they had read the comments and one student did not. (4) Why or why did you 
not read the rubric? Forty-seven students said they read the rubric so they would know how to 
improve their next essay and eighteen said they read the rubric to see where they had made 
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mistakes. The student who did not read the rubric stated that he/she did not think it would be 
useful. Five of the students commented that the section we call ‘Macro feedback’ was the most 
helpful. In the macro feedback section the teaching assistants are required to comment on the 
three major things the student should pay attention to in order to improve their essay. These 
comments are more specific. Ten students felt that the comments on the rubric were not specific 
enough to be helpful. They wanted the teaching assistants to provide examples of how they could 
change their essays. (5) Did you read the comments written on the essay? All eighty-three 
students indicated that they did read the written comments. (6) Why did you read the comments? 
Forty-two students indicated that they wanted to improve their essays therefore they read the 
comments while thirty students said they read the comments to see where they had made 
mistakes so they could fix them. Four students said the comments needed to be more specific. 
From the comments provided it appeared that those students wanted to have exemplars of 
argumentative essays to follow. (7) Students were asked if they had rewritten their essay. 
Seventy-three students said they rewrote their essay and ten did not. In the ARTS 1110 course 
students are allowed to rewrite their 1500 word essays to improve their grade because students 
learn more from rewriting their essay than just reading comments about how they could improve 
the next essay they write. As Spandel (2006) stated there is no growth if students are not 
allowed to use the knowledge they have leamt to improve. (8) Students were then asked why 
they re-wrote or why they did not. Seventy of the students said they rewrote to improve their 
grade, and three students said they rewrote because they had learned how to write a proper essay 
over the course of the term and therefore knew they could improve. For the students who did not 
rewrite their essay, seven of them said that they did not have time to rewrite due to their course 
workload, and one said it was not worthwhile in terms of their overall grade in the course, and 
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two were satisfied with their mark. (9) Did the written comments help you when rewriting your 
essay? Sixty-eight said yes, and eight said no. (10) Why did they? Or why didn’t they? Six 
students said the written comments were more helpful than the comments on the rubric as they 
were specific to their own essay, twelve said the comments showed what was expected in each 
section, Forty students said the comments showed them how to improve their essays and they 
singled out the three macro level comments as being particularly helpful. Of the students who did 
not find the comments helpful they wished for more specific feedback with examples provided. 
(11) What comments would have been helpful when you were rewriting your essay? This 
question was left blank for the most part and those that did answer said the same thing that they 
already said for question number nine so this question was not analyzed in this study. (12) Did 
you go to see your teaching assistant when you got back your marked essay? Forty-three said 
yes and forty said no. (13) Why did you go or why didn’t you go? Of the students who did go, 
forty-two said they wanted to see how they could improve their essay and the other one said 
he/she had seen the instructor instead. Of the students who did not go to see their teaching 
assistant, twenty-one said the rubric and comments provided them with enough guidance to 
know how to improve without going, seven had class during office hours or did not have time 
due to other commitments, and four students said they relied on emails instead of going to talk to 
the teaching assistant. (14) Did you go to see your instructor when you got back your marked 
essay? Twenty-two students went and sixty-one did not. (15) Why did you go or why did you 
not go? Of the students who did go, seventeen went because they wanted to improve, and four 
went to clarify marks. The reasons provided for why they did not go to see their instructor 
included thirty-two who said there was no need to go because their teaching assistants had 
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already shown them how to improve, two had class during office hours, and three had no time 
due to other commitments. 

Experimental Group 

The second survey was administered to the experimental group at the same time as the 
survey provided to the control group which was after the second essay had been marked, and 
handed back to students. The survey consisted of 17 questions. The students were asked which 
rubric they preferred, the analytic or the hybrid holistic. The results are divided into two groups 
and are reported here as analytic group for the students who preferred the analytic rubric and 
holistic group for the students who preferred the hybrid holistic rubric. 

One hundred and one students filled in the second survey. (1) The first question asked 
students to identify which rubric they found the most helpful when they were writing their essay. 
Thirty-three said they preferred the analytic rubric while sixty-nine said they thought the holistic 
rubric was the most helpful. (2) What are some of the features of the rubric, that you chose, that 
you found helpful? Of the students who liked the analytic rubric twenty-nine stated that the 
numbered list was easy to understand, the comments below the numbered list were specific as to 
what was expected, and two said it was easier to follow a list. Of the students who preferred the 
holistic rubric sixty-four students said the letter grade boxes were more useful than the numbered 
list due to the specific criteria that was listed under each category, the comment boxes were 
clearer than the numbered list, they simplified the expectations, they were not just points, and 
they liked the percentages listed as it told them what was needed to obtain a specific grade. (3) 
The third question asked students if there was any other information that could have been 
included on the rubric that would have been helpful. Only 18 students filled in this question and 
the answers were individualized so could not be analyzed. (4) When you received your marked 
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essay back, did you read the rubric for comments provided by the teaching assistant? All thirty- 
three students in the analytic group read the rubric comments and sixty-eight students who liked 
the holistic rubric read the comments. (5) Why did or why did you not read the rubric? Thirty- 
one of the students who preferred the analytic read the comments so they could improve their 
essay and/or identify their errors. Similarly sixty-eight of the students who preferred the holistic 
said they read the comments to either improve their essay or identify the errors they made. (6) 
Did you read the comments written on the essay? In the analytic group all thirty-three read the 
teaching assistant comments and sixty-seven of the holistic group read the comments. (7) Why 
did or why didn’t you read the comments? Twenty-eight of the students in the analytic group 
said they read the comments to improve their next essay and that they found the comments on 
the essay to be more specific to their own paper than the more general comments on the rubric. 

In the holistic group sixty-three of the students said they read the comments to improve their 
essay and to identify errors. (8) Which comments were the most helpful for you when rewriting 
your essay - the rubric or the comments written on the essay? In the analytic group twelve 
students found the comments on the rubrics more helpful while 19 found the comments from the 
teaching assistants more helpful. In the holistic group some students checked both boxes so the 
numbers do not add up to 69. Thirty-two students preferred the comments on the rubric, while 
42 liked the comments written on their essays by the teaching assistants. (9) Did you rewrite 
your essay? In the analytic group twenty-seven said they rewrote their essay and four did not. In 
the holistic group fifty-six rewrote their essay and thirteen did not. (10) Why did you or why did 
you not rewrite your essay? In the analytic group twenty-seven said they rewrote to improve 
their mark, two said they did not rewrite because they were happy with their mark, and two did 
not rewrite because they had no time. In the holistic group fifty-six students rewrote to improve 
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their mark, five indicated that they rewrote because they had learned more about writing over the 
term so they now had a better idea of the requirements of writing an argumentative essay, while 
four said they did not have the time to rewrite due to their course load. (11) Which essay did you 
chose to rewrite? For the analytical group 15 chose to rewrite the first essay and 12 chose to 
rewrite the second essay. In the holistic group, 33 rewrote the first essay and 25 rewrote the 
second essay. (12) Did your choice of which essay to rewrite depend on the rubric provided? In 
the analytic group nine said yes the rubric played a part in their decision while 19 said it made no 
difference. For the holistic group 18 said the decision to rewrite was based on the rubric 
provided while 41 said they did not take into account the rubric when rewriting. (13) Which 
comments were the most useful on the rubric when rewriting your essay? In the analytic group 
only 20 students answered this question and the answers were so varied that no analysis could be 
undertaken. For the holistic group 33 out of 69 students answered this question. Six of the 
students said that the macro level comments helped the most, six said it was the comments that 
were specific to their essay regarding how to improve, and five stated that the comments 
regarding their thesis statement were the most helpful. (14) Are there any comments that would 
have helped you when you were rewriting your essay? Only one student from the analytic group 
answered this question. Similarly in the holistic group only ten students answered the question 
so there is not enough data to draw any conclusions. (15) Did you go to see anyone when you 
got back your marked essay? In the analytic group 16 said they did go for help while 14 said 
they did not. In the holistic group 27 said they did go for help and 39 said they did not go for 
help. (16) Who did you go and see? In the analytic group 14 said they went to see their 
teaching assistant and five went to see the instructor. In the holistic group 22 went to see their 
teaching assistant and eight went to speak to the instructor. (17) Why did you go or why did 
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you not go? In the analytic group eight students went because they wanted to improve their 
essays and six went to clarify comments written on the essay. Eight said that they did not need 
to go because the comments on the rubric and essay provided enough guidance when rewriting 
their essay. For the holistic group eight students went to see how they could improve their essay, 
four went because they wanted to see where they had problems, and ten students went to clarify 
any questions they had about their essay. Twenty students reported that they did not go to see 
anyone because the comments on the rubric and essay were clear enough that they could figure 
out what needed to be done to improve their essays. 

Discussion and Conclusion 

The research presented in this study sought to determine which rubric proved most 
beneficial to students when rewriting their argumentative essay, the analytic rubric or the hybrid 
holistic rubric. It emerged that while some students had a preference for the type of rubric used, 
both rubrics were equally effective in providing information to the students regarding writing 
their essays and rewriting after they received feedback. The main benefits of the rubrics for the 
students were that the rubrics provided clear guidelines and expectations for the assignments. 

This is in keeping with Sadler’s (2009) assertion that the purpose for rubrics is to provide clear 
expectations for the students. The rubrics allowed students to understand what they needed to 
adjust when writing their essays. As Moskal (2000) stated rubrics can be said to be beneficial if 
they provide feedback to students on how to improve their performance. It appears that the two 
rubrics used in the ARTS 1110 class fulfilled that criteria. 

Our research questions were “Can first-year students taking the Arts 1110 Introduction to 
University course articulate their understanding of the marking rubric? and, “How do students 
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use their understanding of the marking rubric when they are revising their academic essays in 
ARTS 1110?” 

The comments provided by the students indicate that they were able to articulate their 
understanding of what the purpose of the rubrics were and how to use the rubrics to improve 
their essays. 

Analytic Rubric 

The students who were in the control group had both of their essays marked using the 
analytic rubric. Seventy-one percent found the rubric helpful when writing their essay because it 
showed them what needed to be included in their essays, the criteria was specific, and it provided 
a guideline as to what the markers were looking for when grading their essays. Fifty-seven 
percent said they used the rubric comments to improve their essays and to see where they made 
mistakes. Eighty-eight percent of the students took the opportunity to rewrite their essays 
because they wanted to improve their grades. 

In our study, 82% of the students in the experimental group said that the analytic rubric 
provided was useful because it provided clear expectations of what was expected in their essays, 
helped with the organization of the essay, and provided a checklist of what was expected by the 
markers. Ninety-one percent of the students said they read the comments provided by the 
teaching assistants for how to improve their essays or make corrections. When students were 
asked if they went to see their teaching assistants after getting their marked essay back, only 37% 
of the students reported going to see their teaching assistants while 62% did not go. The students 
who went to see their teaching assistants said they went to clarify comments so they could 
improve their mark. The main reason why the students did not go to see their teaching assistant 
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was that the rubrics and comments on the essay provided guidelines that were clear enough so 
they did not see a need to go for extra help. 

Hybrid Holistic Rubric 

The experimental group had their second essays marked using the hybrid holistic rubric 
which had letter grades across the top with the corresponding criteria under each letter grade. Of 
the 101 students who filled in the survey 68% of them preferred the holistic rubric over the 
analytic rubric because they said the letter boxes were more useful than the numbered list due to 
the specific criteria that was listed under each category, the boxes simplified the expectations, 
and they liked the percentages being listed because it told them what was needed to get a specific 
grade. Both groups, the ones who preferred the analytic and the ones who preferred the holistic 
cited the comments helped them improve their essays and helped them identify the errors they 
made in their essays. They also said they preferred the comments written on their essays rather 
than the rubrics because they were more specific to their own essay. The same percentage of 
students in the analytic group and the holistic group rewrote their essays because they wanted to 
improve their essays. They admitted that the rubrics, for the most part, did not enter into their 
decision as to which essay they chose to rewrite. The decision was based on which essay had the 
lower grade because they wanted to be able to improve their mark. Again, the majority of the 
students did not go to either the teaching assistant or the instructor for help because they felt that 
the comments on the rubric and the comments on the essay were clear enough that they knew 
what had to be done to improve their essay. 

While the research presented in this paper indicates that for students who had a choice, 
they preferred the holistic rubric because they liked the specific criteria listed under the letter 
grades so they were aware of what they needed to do to receive an A on the essay, the analytic 
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rubric was also well received. The answers provided on the surveys indicate that students come 
to university without clear expectations of what will be required of them in terms of writing an 
argumentative essay. The rubrics provided those expectations and also explained what criteria 
was used in each category. The findings indicate that the students have an understanding of the 
purpose of the rubrics. They understand that the rubrics are not just useful to see what mark they 
received but to also guide them when rewriting their essays. While the majority of the students 
were most concerned with getting higher grades there were also many comments about how the 
rubrics helped them improve their essays. This is confirmed by the data in this study where 82% 
of the students indicated that they had learned how to improve their essays based on the 
comments provided on the rubrics. 

The purpose for conducting the study was to find out if one rubric was more beneficial to 
students when they were rewriting their essay. The data indicates that the rubric does not matter 
as much as the clarity of what is expected from the student. This study provides evidence that as 
instructors we have to be aware that when students read rubrics they are looking for comments 
that tell them where they went wrong and how to improve. Some of the students’ comments 
included, “it is more helpful to find out what you need to work on then what you did right. And, 
it is important that we learn how to write an essay in first-year so that we can use that knowledge 
in later years. We are old enough to take constructive criticism and that is more important than 
being told we did something well”. Some students commented that the marks they received were 
what they deserved and that they expected the marks since they did not put as much effort into 
writing the essay as they should have. They said the rubrics were helpful when rewriting as they 
appreciated the comments explaining how they could improve their essays. 
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The findings of this study indicated that as instructors in order to construct a reliable 

rubric we have to take into account our expectations of the assignment. Once we can articulate 

what it is we expect our students to be able to do after writing their essays than we can construct 

a rubric that covers our expectations. Another thing to keep in mind is the language we use on 

the rubrics. Some of the students indicated that they had trouble understanding exactly what it 

was they were expected to do based on the comments provided on the rubrics and by the 

teaching assistants. This indicates that we have to keep in mind not only the purpose of the 

rubric but also the audience. It is only by keeping in mind those two factors that instructors can 

design a rubric that fits not only the expectations of the instructors but those of the students as 

well. One of the recommendations that came out of this study is to make university instructors 

aware that if they expect students to meet their course expectations than the students need to 

understand not only the expectations but the terms that will be used. 

Limitations 

One of the limitations of the study is that only one course was surveyed in one university 
to gather the data. However, based on the review of literature, many of the findings are 
supported by the findings of other research based studies concerning rubrics. 
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Appendix A Analytic Rubric 


Name: 


ARTS 1110 - Introduction to University /100 

Essay# Lab# 


Introduction and Thesis 


0 


2 3 4 5 6 7 8 9 10 11 12 13 14 15 

□ Thesis is not clearly stated and must be constructed from the text of the paper. Make sure your 
reader knows your stance (what side of the argument you are on) and the three reasons for your 
stance. 

□ Does not demonstrate an understanding of the topic by providing enough detail to orientate the 
reader. Expand on the topic by providing some background information on the topic. 

□ Relevant terms are not defined in the introduction with scholarly evidence. Just define the topic 
not all of the terms in the essay. You can define the other terms in the body paragraphs. 

□ Other comments: 


Argument and Reasoning 

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 

□ Does not show evidence of critical thought and analysis of the topic. 

□ Your essay requires a clear argument not just a description of facts. Remember that you are trying 
to convince someone that your argument is valid. Just providing facts will not convince your 
reader. 

□ Your argument is weak because you do not have scholarly evidence to support your claims. 
Remember that personal opinion is not convincing. To improve add in evidence from scholarly 
articles or books. 

□ Your counter argument is weak because it is not a plausible alternative explanation for your 
argument and/or it is not supported with scholarly evidence. 

□ Main points are not sufficiently supported with evidence to support your claims or are incomplete, 
incorrect, or oversimplified. 

□ Other comments: 


Pattern of Organization 


0 1 2 3 4 5 6 7 


9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 


□ Paper is not organized sequentially and argument is difficult to follow. 

□ Paragraphs are not consistently well developed, with a clear topic sentence. Keep in mind that 
your topic sentence should introduce what you are going to be discussing in the paragraph. 

□ Each claim is not synthesized with a summary statement. A summary statement is a mini 
conclusion for each claim. 

□ Paragraphs do not follow the pattern of organization established in the thesis statement. Discuss 
claims in the same order as your thesis statement. 

□ Ensure that your paragraphs all work to support the thesis statement and have clear transitions 
throughout. 

□ Revise paragraphs to ensure that each new idea is developed in a separate paragraph. 

□ Other comments: 
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Conclusion 01234567 8 9 10 


□ Your argument has just been summarized and not synthesized, so the reader is left to draw 
their own conclusions. Your reader should understand the argument that has been discussed. 

□ The evidence used to support the claims has not been articulated. Do not restate evidence but 
instead state the main points of the evidence with the citation. 

□ Do not simply rewrite the thesis statement. Instead discuss each claim separately with the 
evidence used to support that claim. 

□ The claims are not reported in the same order as thesis statement. Keep the pattern of 
organization from your thesis statement. 

□ Counter-argument has not been acknowledged in the beginning of the conclusion 

□ The conclusion is missing 

□ Other comments: 


Resources/Research 0 1 2 3 4 56789 10 

□ No reference page 

Cited material is not relevant, reliable or authoritative (e.g. scholarly journals). 

□ One or more references are not scholarly. 

□ Annotation does not summarize the articles in enough detail so the reader understands the main 
points. 

□ Writer does not clearly explain how the article will be used to verify the validity of the claims or 
the counter- argument. 

□ No annotated bibliography 

□ Other comments: 


Format and Style (APA 6 th Edition) See chapter 12 of textbook for correct APA formatting or the OWL at 
Purdue website. 0 1 2 3 4 5 


fj Title page does not follow UM Learn template 

□ Pages do not follow APA 6 format (Running head, margins, spacing, etc. . .) 

□ In-text citations do not adhere to APA format 

□ Reference page does not adhere to APA format 

□ Language - third person language is not used throughout essay 

□ Other comments: 


Grammar and Composition Total: /10 

a. Grammar, syntax, spelling, punctuation /5 

□ 1 Incomprehensible 

□ 2 Difficult to understand 

□ 3-4 Understandable with persistent errors 

□ 5 Few errors 

b. Diction - precision and clarity of writing. /5 

□ Word choice is not appropriate for formal essay writing 

□ Avoid inappropriate use of colloquial/slang/informal language 

□ Avoid the use of contractions 

□ Avoid the use of abbreviations 
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Macro Level Feedback - Concentrate on Revising the Following Three Things 


1 . 


2 . 


3 . 


Other General Comments 
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Appendix B Hybrid Holistic Rubric 

Holistic Marking Rubric for ARTS 1110 Essays 



The A Paper 

The B Paper 

The C Paper 

The D Paper 

The F Paper 

Introduction/ 

Thesis 

Thesis is clear 
and concise. 
Explains topic in 
enough detail 
for the reader to 
be able to tell 
what the paper 
will be about 
and defines 
relevant terms. 

Thesis is clear but 
does not explain 
topic in detail 
and/or define 
relevant terms. 

Thesis is in the 
introduction 
but not clear. 
Topic needs 
clarification 
and terms need 
defining. 

No clear thesis 
statement. The 
reader has a vague 
idea of what the 
essay is about but 
has to search for the 
argument or the 
claims. Topic is not 
clear and terms are 
not defined. 

No clear thesis 
statement. It is 
difficult to 
understand what 
the writer is going 
to be arguing. 
Topic is not 
explained and 
terms are not 
defined. 

Argument 

and 

Reasoning 

Consistently 
applies a high 
level of critical 
thinking. 

Argument is well- 
supported and 
convincing. Fairly 
represents 
opposing views. 

Argument is 
logical but may be 
missing some key 
evidence. 
Counter-argument 
is logical and 
supported with 
evidence. 

Some aspects of 
the argument are 
not relevant to 
the claims being 
expressed OR 
the evidence 
does not support 
the claims. 
Counter- 
argument may 
not be fully 
supported. 

Argument has some 
relevant points but 
overall argument is 
weak due to lack of 
support/evidence. 
May not consider a 
counter-argument or 
it is not valid. 

Argument is not 
well-supported 
with scholarly 
evidence and/or 
is not 

convincing. No 
counter- 
argument. 

Organization 

Follows the 
pattern of 
organization 
dictated by the 
thesis statement. 
Paragraphs are 
well-developed. 
Strong 

transitions, topic 
sentences, 
summary 
statements. 

Follows the 
pattern of 
organization 
dictated by the 
thesis statement. 
Paragraphs are not 
as well-developed 
with topic 
sentences, 
transitions and 
summary 
statements. 

Pattern of 
organization 
follows the 
thesis 
statement. 
Paragraphs are 
not fully 
developed to 
include the 
claim and the 
evidence to 
support that 
claim. 

Does not follow 
pattern of 
organization based 
on thesis 
statement (or 
thesis statement 
was not clear). 
Paragraphs are not 
fully developed. 
No topic 
sentences. No 
summary 
statements. 

Does not follow 
any pattern of 
organization due 
to no thesis 
statement. 
Paragraphs are 
weak with no 
clear topic 
sentence, and no 
summary 
statement. 

Conclusion 

Summarizes and 
synthesizes the 
claims and 
provides the 
evidence found 
to support those 
claims. Claims 
are reported in 
same order as 
thesis statement. 

Summarizes the 
claims but does 
not synthesize the 
claims with 
evidence to 
support the topic. 

Summarizes 
the claims but 
may not be in 
the same order 
as the essay. 
Writer does not 
indicate how 
they proved the 
claims. 

Restates the 
introduction but 
makes an attempt 
to synthesize the 
evidence. 

Simply restates 
the introduction. 
No attempt to 
summarize and 
evaluate the 
claims. 

OR there is no 
conclusion 
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Research 

Proficient use 
of references. 
The references 
are reliable and 
authoritative. 
Annotation 
adequately 
summarizes 
the article and 
indicates 
which claim it 
supports. 

Good use of 
reliable sources 
but one or more 
are not 
authoritative. 
Annotations are 
complete but 
may not indicate 
which claim 
they support. 

Some 

references are 
scholarly but 
there are 
many that are 
not. 

There are 
annotations 
but they do 
not 

adequately 
summarize 
the article 
and/or they 
do not 
indicate 
which claim 
they support. 

References used 
are taken from 
questionable 
sources with 
only one or two 
that are 
scholarly. 

May have 
missed one or 
more of the 
annotations 
and/or the 
annotations 
presented are not 
complete nor do 
they indicate 
which claim 
they support. 

References are 
not scholarly 
OR there is no 
reference page. 
No annotated 
Bibliography 

Grammar 

and 

Composition 

Almost 

entirely free of 
spelling, 
punctuation 
and 

grammatical 
errors. Word 
choice is 
appropriate for 
clarity of 
meaning. 

May contain a 
few errors that 
do not impede 
understanding. 
Some words are 
used incorrectly. 

Mechanical 
errors may 
confuse but 
do not detract 
from overall 
meaning. 

May use 
some words 
that are not 
appropriate. 

Contains many 
grammatical 
errors that 
impede 
meaning. 
Problems with 
appropriate word 
choice. 

Grammatical 
errors detract 
from the 
meaning. Poor 
word choice. 

APA 

Formatting 

Title Page, All 
pages follow 
APA 

Formatting, 
References 
Page, Intext 
Citations are 
correct 

One of the 
elements is 
incorrect. 

Two elements 
are incorrect. 

Three elements 
are incorrect. 

Major errors in 
APA 

formatting. 
Check title 
page, running 
head, 
references 
page, intext 
citations. 

Mark 

80% - 100% 

70% - 79.9% 

60% - 69.9% 

50% - 59.9% 

Less than 50 


Macro Level Feedback - Concentrate on Revising the Following Three Things: 


Other General Comments 
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Appendix C Analytic Rubric - Experimental Group 

Student Survey 
Winter 2015 

ARTS 1110 Introduction to University 

We are gathering data about the rubric used to mark your essays in this course. Think back to 
when you first received the rubric in class, and then when you got your marked essay back, with 
the rubric, and answer the following questions. 

1. Did the rubric prove helpful when you were writing your essay? Yes No 

2. Why was it helpful? Or Why was it not helpful? Please elaborate. 


3. When you received your marked 500 word essay back, did you read the rubric for the 

comments provided by the Teaching Assistants? Yes 

No 

4. Why did or why did you not read the rubric? Were the comments on the rubric helpful? 
Please elaborate. 


5. Did you read the comments written on the essay? Yes No 

6. Why did or why didn’t you read the comments? Please elaborate. 


7. Did you go to see your Teaching Assistant when you got back your marked essay? Yes No 
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8. Why did you go? Or why did you not go? Please elaborate. 
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9. Did you go to see your instructor when you got back your marked essay? Yes No 

10. Why did you go? Or why did you not go? Please elaborate. 


Please provide any other information that we might find useful in assessing the usefulness of the 
marking rubric or the comments on the essay itself. Thank you. 
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Appendix D Analytic Rubric Control Group 

Student Survey 
Winter 2015 

ARTS 1110 Introduction to University 

We are gathering data about the rubric used to mark your essays in this course. Think back to 
when you first received the rubric in class, and then when you got your marked essay back, with 
the rubric, and answer the following questions. 

1. Did the rubric prove helpful when you were writing your essay? Yes No 

2. Why was it helpful? Or Why was it not helpful? Please elaborate. 


3. When you received your marked essay back, did you read the rubric for the comments 

provided by the Teaching Assistants? Yes No 

4. Why did or why did you not read the rubric? Were the comments on the rubric helpful? 
Please elaborate. 


5. Did you read the comments written on the essay? Yes No 

6. Why did or why didn’t you read the comments? Please elaborate. 


7. Did you rewrite your essay? 


Yes 


No 
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8. Why did you? Or Why did you not rewrite? Please elaborate. 
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9. Did the written comments in the essay help you when rewriting your essay? Yes No 

10. Why did they? Or why didn’t they? Please elaborate. 


11. What comments would have helped you when you were rewriting your essay? 


12. Did you go to see your Teaching Assistant when you got back your marked essay? Yes No 

13. Why did you go? Or why did you not go? Please elaborate. 


14. Did you go to see your instructor when you got back your marked essay? Yes No 

15. Why did you go? Or why did you not go? Please elaborate. 


Please provide any other information that we might find useful in assessing the usefulness of the 
marking rubric or the comments on the essay itself. Thank you. 
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Appendix E Experimental Group Comparison between the Analytic and Hybrid Holistic Rubric 

Student Survey 
Winter 2015 

ARTS 1110 Introduction to University 

We are gathering data about the rubrics used to mark your essays in this course. Think back to 
when you first received the rubrics in class, and then when you got your marked essay back, with 
the rubric, and answer the following questions. 

1. Which rubric proved the most helpful when you were writing your essay? (check one) 

□ The first one with the numbered list for each section 

□ The second one with the Letter Grades across the top 

2. What were some features of the rubric, that you chose in the question above, that you found 
helpful? Please elaborate. 


3. Is there any other information you wish had been included on the rubric you found the most 
helpful? Please elaborate. 


4. When you received your marked essay back, did you read the rubric for the comments 

provided by the Teaching Assistants? Yes No 

5. Why did or why did you not read the rubric? Were the comments on the rubric helpful? 
Please elaborate. 


6. Did you read the comments written on the essay? 

7. Why did or why didn’t you read the comments? Please elaborate. 


Yes 


No 
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8. Which comments were the most helpful for you? Check one. 

□ The comments on the rubrics. 

□ The comments on the essay. 

9. Did you rewrite your essay? Yes No 

10. Why did you? Or Why did you not rewrite? Please elaborate. 


11. Which essay did you chose to rewrite? Check one. 

□ The first essay 

□ The second essay 

12. Did your choice of which essay to rewrite depend on the rubric provided? Yes No 

13. Which comments were the most useful on the rubric when you were rewriting your essay? 
Please elaborate. 


14. Are there any comments that would have helped you when you were rewriting your essay? 
Please elaborate. 


15. Did you go to see anyone when you got back your marked essay? Yes No 

16. Who did you go and see? Check all that apply. 

□ The teaching assistant 

□ The instructor 

17. Why did you go? Or why did you not go? Please elaborate. 


Please provide any other information that we might find useful in assessing the usefulness of the 
marking rubric or the comments on the essay itself. Thank you. 



